An Image Classification Example: Dolphin v.s. Giraffe

In [1]:
%matplotlib inline
In [2]:
import os 
host = os.environ['CASHOST']
port = os.environ['CASPORT']
userid = None
password = None
In [3]:
# needed to start a CAS server.
from swat import * 
sess = CAS(host, port,userid, password)
In [4]:
from dlpy.images import ImageTable

Load images

In [5]:
my_images = ImageTable.load_files(sess, path='/dept/cas/leliuz/Data/Demo/Giraffe_Dolphin')
In [6]:
my_images.head()
Out[6]:
Selected Rows from Table IMAGEDATA_EJXS02
_image_ _label_ _filename_0
0 b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01... Dolphin dolphin_10158.jpg
1 b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01... Dolphin dolphin_10920.jpg
2 b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01... Dolphin dolphin_10924.jpg
3 b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01... Dolphin dolphin_10213.jpg
4 b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01... Dolphin dolphin_10937.jpg
In [7]:
my_images.columninfo()
Out[7]:
§ ColumnInfo
Column ID Type RawLength FormattedLength NFL NFD
0 _image_ 1 varbinary 1075587 1075587 0 0
1 _label_ 2 varchar 7 7 0 0
2 _filename_0 3 varchar 17 17 0 0

elapsed 0.0105s · user 0.00921s · sys 0.0159s · mem 2.93MB

In [8]:
my_images.show(nimages=8,ncol=4, randomize=True)
In [9]:
my_images.label_freq
Out[9]:
Frequency for IMAGEDATA_EJXS02
Level Frequency
Dolphin 1 237
Giraffe 2 172
In [10]:
my_images.image_summary
Out[10]:
jpg                   409
minWidth              170
maxWidth             1024
minHeight             127
maxHeight            1024
meanWidth         912.147
meanHeight        778.499
mean1stChannel    128.718
min1stChannel           0
max1stChannel         255
mean2ndChannel    124.123
min2ndChannel           0
max2ndChannel         255
mean3rdChannel    94.2449
min3rdChannel           0
max3rdChannel         255
dtype: object

Image processing

In [11]:
my_images.resize(width=224)
my_images.show(8,4)

Splitting the data into training and testing

In [12]:
from dlpy.splitting import two_way_split
In [13]:
tr_img, te_img = two_way_split(my_images,test_rate=20, seed = 123)
In [14]:
tr_img.label_freq
Out[14]:
Frequency for IMAGEDATA_WN2BSY
Level Frequency
Dolphin 1 190
Giraffe 2 138
In [15]:
te_img.label_freq
Out[15]:
Frequency for IMAGEDATA_QSTOME
Level Frequency
Dolphin 1 47
Giraffe 2 34

Training Data Augmentation

In [16]:
tr_img.as_patches(width=200,height=200,step_size=24,output_width=224,output_height=224)
In [17]:
tr_img.label_freq
Out[17]:
Frequency for IMAGEDATA_WN2BSY
Level Frequency
Dolphin 1 760
Giraffe 2 552
In [18]:
tr_img.show(8,4)

Model1: Simple CNN

Build a simple CNN model

In [19]:
from dlpy import Model, Sequential
from dlpy.layers import * 
from dlpy.applications import *
In [20]:
model1 = Sequential(sess, model_name = 'Simple_CNN')

Input Layer

In [21]:
model1.add(InputLayer(3,224,224,offsets=tr_img.channel_means))

NOTE: Input layer added.

1 Convolution 1 Pooling

In [22]:
model1.add(Conv2d(8,7))
model1.add(Pooling(2))

NOTE: Convolutional layer added.
NOTE: Pooling layer added.

1 Convolution 1 Pooling

In [23]:
model1.add(Conv2d(8,7))
model1.add(Pooling(2))

NOTE: Convolutional layer added.
NOTE: Pooling layer added.

1 Dense layer

In [24]:
model1.add(Dense(16))

NOTE: Fully-connected layer added.

Output layer

In [25]:
model1.add(OutputLayer(act='softmax',n=2))

NOTE: Output layer added.
NOTE: Model compiled successfully.
In [26]:
model1.print_summary()

*==================*===============*========*============*=================*======================*
|   Layer (Type)   |  Kernel Size  | Stride | Activation |   Output Size   | Number of Parameters |
*------------------*---------------*--------*------------*-----------------*----------------------*
| Data(Input)      |     None      |  None  |    None    |  (224, 224, 3)  |        0 / 0         |
| Conv1_1(Convo.)  |    (7, 7)     |   1    |    Relu    |  (224, 224, 8)  |       1176 / 8       |
| Pool1(Pool)      |    (2, 2)     |   2    |    Max     |  (112, 112, 8)  |        0 / 0         |
| Conv2_1(Convo.)  |    (7, 7)     |   1    |    Relu    |  (112, 112, 8)  |       3136 / 8       |
| Pool2(Pool)      |    (2, 2)     |   2    |    Max     |   (56, 56, 8)   |        0 / 0         |
| FC1(F.C.)        |  (25088, 16)  |  None  |    Relu    |       16        |     401408 / 16      |
| Output(Output)   |    (16, 2)    |  None  |  Softmax   |        2        |        32 / 2        |
*==================*===============*========*============*=================*======================*
|Total Number of Parameters: 405,786                                                              |
*=================================================================================================*
In [27]:
# You need install graphviz to run this, otherwise, just skip this line.
model1.plot_network()
Out[27]:
Simple_CNN DAG for Simple_CNN: Data Data (input) Input Size: (224, 224, 3) Conv1_1 Conv1_1 (convo) Kernel Size: Activation: Output Size: (7, 7) Relu (224, 224, 8) Data->Conv1_1 Pool1 Pool1 (pool) Kernel Size: Output Size: (2, 2) (112, 112, 8) Conv1_1->Pool1 Conv2_1 Conv2_1 (convo) Kernel Size: Activation: Output Size: (7, 7) Relu (112, 112, 8) Pool1->Conv2_1 Pool2 Pool2 (pool) Kernel Size: Output Size: (2, 2) (56, 56, 8) Conv2_1->Pool2 FC1 FC1 (fc) Kernel Size: Activation: Output Size: (25088, 16) Relu 16 Pool2->FC1 Output Output (output) Kernel Size: Activation: Output Size: (16, 2) Softmax 2 FC1->Output

Model training

In [28]:
model1.fit(data=tr_img, mini_batch_size=2, max_epochs=3, lr=1E-4)

NOTE: Training from scratch.
NOTE:  The Synchronous mode is enabled.
NOTE:  The total number of parameters is 405786.
NOTE:  The approximate memory cost is 1425.00 MB.
NOTE:  Loading weights cost       0.00 (s).
NOTE:  Initializing each layer cost       0.76 (s).
NOTE:  The total number of workers is 7.
NOTE:  The total number of threads on each worker is 32.
NOTE:  The total minibatch size per thread on each worker is 2.
NOTE:  The maximum minibatch size across all workers for the synchronous mode is 448.
NOTE:  Target variable: _label_
NOTE:  Number of levels for the target variable:      2
NOTE:  Levels for the target variable:
NOTE:  Level      0: Dolphin
NOTE:  Level      1: Giraffe
NOTE:  Number of input variables:     1
NOTE:  Number of numeric input variables:      1
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              448  0.0001          10.817      0.5915         0.03
NOTE:        1              448  0.0001          9.5084      0.4129         0.02
NOTE:        2              382  0.0001           0.607      0.3246         0.02
NOTE:        3               34  0.0001          0.6833      0.4118         0.02
NOTE:  Epoch           Learning Rate     Loss    Fit Error      Time (s)
NOTE:          0          0.0001       7.1347     0.4482         2.16
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              448  0.0001          0.6425      0.2835         0.05
NOTE:        1              448  0.0001          0.6014      0.2455         0.02
NOTE:        2              382  0.0001           0.557      0.2644         0.05
NOTE:        3               34  0.0001          0.6045      0.3529         0.02
NOTE:          1          0.0001       0.6026     0.2668         2.18
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              448  0.0001           0.601        0.25         0.03
NOTE:        1              448  0.0001          0.5644      0.2143         0.04
NOTE:        2              382  0.0001          0.5289      0.2356         0.04
NOTE:        3               34  0.0001          0.5717      0.3235         0.06
NOTE:          2          0.0001       0.5667     0.2355         2.88
NOTE:  The optimization reached the maximum number of epochs.
NOTE:  The total time is       7.22 (s).
Out[28]:
§ ModelInfo
Descr Value
0 Model Name simple_cnn
1 Model Type Convolutional Neural Network
2 Number of Layers 7
3 Number of Input Layers 1
4 Number of Output Layers 1
5 Number of Convolutional Layers 2
6 Number of Pooling Layers 2
7 Number of Fully Connected Layers 1
8 Number of Weight Parameters 405752
9 Number of Bias Parameters 34
10 Total Number of Model Parameters 405786
11 Approximate Memory Cost for Training (MB) 1425
§ OptIterHistory
Epoch LearningRate Loss FitError
0 1 0.0001 7.134707 0.448171
1 2 0.0001 0.602615 0.266768
2 3 0.0001 0.566738 0.235518
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Simple_CNN_weights 405786 3 CASTable('Simple_CNN_weights', caslib='CASUSER...

elapsed 8.29s · user 727s · sys 25.9s · mem 1.04e+04MB

In [29]:
model1.fit(data=tr_img, mini_batch_size=3, max_epochs=3, lr=5E-5, log_level=3)

NOTE: Training based on existing weights.
NOTE:  The Synchronous mode is enabled.
NOTE:  The total number of parameters is 405786.
NOTE:  The approximate memory cost is 1612.00 MB.
NOTE:  Loading weights cost       0.03 (s).
NOTE:  Initializing each layer cost       0.52 (s).
NOTE:  The total number of workers is 7.
NOTE:  The total number of threads on each worker is 32.
NOTE:  The total minibatch size per thread on each worker is 3.
NOTE:  The maximum minibatch size across all workers for the synchronous mode is 672.
NOTE:  Target variable: _label_
NOTE:  Number of levels for the target variable:      2
NOTE:  Levels for the target variable:
NOTE:  Level      0: Dolphin
NOTE:  Level      1: Giraffe
NOTE:  Number of input variables:     1
NOTE:  Number of numeric input variables:      1
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              672    5E-5          0.5717      0.2158         0.06
NOTE:        1              606    5E-5          0.5148      0.2195         0.07
NOTE:        2               34    5E-5          0.5509      0.2647         0.03
NOTE:  Epoch           Learning Rate     Loss    Fit Error      Time (s)
NOTE:          0            5E-5       0.5449     0.2188         2.44
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              672    5E-5          0.5633      0.2098         0.02
NOTE:        1              606    5E-5          0.5063      0.2162         0.02
NOTE:        2               34    5E-5          0.5408      0.2941         0.03
NOTE:          1            5E-5       0.5364     0.2149         2.02
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              672    5E-5          0.5559      0.2054         0.02
NOTE:        1              606    5E-5           0.499      0.2079         0.04
NOTE:        2               34    5E-5          0.5312      0.2941         0.03
NOTE:          2            5E-5        0.529     0.2088         1.91
NOTE:  The optimization reached the maximum number of epochs.
NOTE:  The total time is       6.37 (s).
Out[29]:
§ ModelInfo
Descr Value
0 Model Name simple_cnn
1 Model Type Convolutional Neural Network
2 Number of Layers 7
3 Number of Input Layers 1
4 Number of Output Layers 1
5 Number of Convolutional Layers 2
6 Number of Pooling Layers 2
7 Number of Fully Connected Layers 1
8 Number of Weight Parameters 405752
9 Number of Bias Parameters 34
10 Total Number of Model Parameters 405786
11 Approximate Memory Cost for Training (MB) 1612
§ OptIterHistory
Epoch LearningRate Loss FitError
0 4 0.00005 0.544852 0.218750
1 5 0.00005 0.536396 0.214939
2 6 0.00005 0.528979 0.208841
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Simple_CNN_weights 405786 3 CASTable('Simple_CNN_weights', caslib='CASUSER...

elapsed 7.25s · user 767s · sys 21.8s · mem 1.17e+04MB

Summerize the training history

In [30]:
model1.training_history
Out[30]:
Epoch LearningRate Loss FitError
0 1 0.00010 7.134707 0.448171
1 2 0.00010 0.602615 0.266768
2 3 0.00010 0.566738 0.235518
3 4 0.00005 0.544852 0.218750
4 5 0.00005 0.536396 0.214939
5 6 0.00005 0.528979 0.208841
In [31]:
model1.plot_training_history(fig_size=(10,4))

Scoring

In [32]:
model1.predict(te_img)
Out[32]:
§ ScoreInfo
Descr Value
0 Number of Observations Read 81
1 Number of Observations Used 81
2 Misclassification Error (%) 20.98765
3 Loss Error 0.546549
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Valid_Res_Sj2beY 81 8 CASTable('Valid_Res_Sj2beY', caslib='CASUSERHD...

elapsed 0.704s · user 5.46s · sys 3.63s · mem 8.9e+03MB

In [33]:
model1.valid_conf_mat
Out[33]:
§ Crosstab
_label_ Col1 Col2
0 Dolphin 36.0 11.0
1 Giraffe 6.0 28.0

elapsed 0.04s · user 0.0595s · sys 0.0714s · mem 13.1MB

In [34]:
model1.plot_predict_res(type='C',image_id=2)
In [35]:
model1.plot_predict_res(type='M',image_id=0)

Heatmap analysis

In [36]:
model1.heat_map_analysis(data=te_img, mask_width=56, mask_height=56, step_size=8)

NOTE : The number of images in the table is too large, only 5 randomly selected images are used in analysis.
NOTE: Table IMAGEDATA_KETQS0 contains compressed images.
Out[36]:
_filename_0 _image_ _label_ heat_map
0 giraffe_11069.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Giraffe [[0.5000260472297668, 0.5000260472297668, 0.50...
1 giraffe_10831.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Giraffe [[0.5000260472297668, 0.5000260472297668, 0.50...
2 dolphin_10937.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.7850251197814941, 0.7850251197814941, 0.78...
3 dolphin_10368.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.9709257483482361, 0.9709257483482361, 0.97...
4 dolphin_10811.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.49997398257255554, 0.49997398257255554, 0....

View feature maps

In [37]:
model1.get_feature_maps(data=te_img,label='Dolphin', image_id=0)
In [38]:
model1.feature_maps.display(layer_id=0)
In [39]:
model1.feature_maps.display(layer_id=1)
In [40]:
model1.feature_maps.display(layer_id=2)

Save model to astore

In [41]:
model1.deploy(path='/dept/cas/leliuz/Data/Demo/', output_format='astore')

NOTE: Model astore file saved successfully.
In [42]:
model1.deploy(path='/dept/cas/leliuz/Data/Demo/', output_format='table')

NOTE: Model table saved successfully.

Model2: ImageNet ResNet50

In [43]:
from dlpy.applications import ResNet50_Caffe, VGG16
In [44]:
model2 = ResNet50_Caffe(sess, model_name='RESNET50_CAFFE', batch_norm_first=False,
                   n_classes=1000, n_channels=3, width=224, height=224, scale=1,
                   random_flip='none', random_crop='none',
                   offsets=(103.939, 116.779, 123.68),
                   pre_train_weight=True, include_top=True)

NOTE: Model table is attached successfully!
NOTE: Model is named to "resnet50_caffe" according to the model name in the table.
NOTE: Cloud Analytic Services made the uploaded file available as table LABEL_ZKN3CO in caslib CASUSERHDFS(leliuz).
NOTE: The table LABEL_ZKN3CO has been created in caslib CASUSERHDFS(leliuz) from binary data uploaded to Cloud Analytic Services.
In [45]:
model2.print_summary()

*==================*===============*========*============*=================*======================*
|   Layer (Type)   |  Kernel Size  | Stride | Activation |   Output Size   | Number of Parameters |
*------------------*---------------*--------*------------*-----------------*----------------------*
| data(Input)      |     None      |  None  |    None    |  (224, 224, 3)  |        0 / 0         |
| conv1(Convo.)    |    (7, 7)     |   2    |  Identity  | (112, 112, 64)  |      9408 / 64       |
| bn_conv1(B.N.)   |     None      |  None  | Rectifier  | (112, 112, 64)  |       0 / 128        |
| pool1(Pool)      |    (3, 3)     |   2    |    Max     |  (56, 56, 64)   |        0 / 0         |
| res2a_branch2a...|    (1, 1)     |   1    |  Identity  |  (56, 56, 64)   |       4096 / 0       |
| bn2a_branch2a(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2a_branch2b...|    (3, 3)     |   1    |  Identity  |  (56, 56, 64)   |      36864 / 0       |
| bn2a_branch2b(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2a_branch2c...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2a_branch2c(...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2a_branch1(...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2a_branch1(B...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2a(Resid.)    |     None      |  None  | Rectifier  |  (56, 56, 256)  |        0 / 0         |
| res2b_branch2a...|    (1, 1)     |   1    |  Identity  |  (56, 56, 64)   |      16384 / 0       |
| bn2b_branch2a(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2b_branch2b...|    (3, 3)     |   1    |  Identity  |  (56, 56, 64)   |      36864 / 0       |
| bn2b_branch2b(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2b_branch2c...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2b_branch2c(...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2b(Resid.)    |     None      |  None  | Rectifier  |  (56, 56, 256)  |        0 / 0         |
| res2c_branch2a...|    (1, 1)     |   1    |  Identity  |  (56, 56, 64)   |      16384 / 0       |
| bn2c_branch2a(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2c_branch2b...|    (3, 3)     |   1    |  Identity  |  (56, 56, 64)   |      36864 / 0       |
| bn2c_branch2b(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2c_branch2c...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2c_branch2c(...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2c(Resid.)    |     None      |  None  | Rectifier  |  (56, 56, 256)  |        0 / 0         |
| res3a_branch2a...|    (1, 1)     |   2    |  Identity  |  (28, 28, 128)  |      32768 / 0       |
| bn3a_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3a_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3a_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3a_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3a_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3a_branch1(...|    (1, 1)     |   2    |  Identity  |  (28, 28, 512)  |      131072 / 0      |
| bn3a_branch1(B...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3a(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res3b_branch2a...|    (1, 1)     |   1    |  Identity  |  (28, 28, 128)  |      65536 / 0       |
| bn3b_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3b_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3b_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3b_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3b_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3b(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res3c_branch2a...|    (1, 1)     |   1    |  Identity  |  (28, 28, 128)  |      65536 / 0       |
| bn3c_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3c_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3c_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3c_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3c_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3c(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res3d_branch2a...|    (1, 1)     |   1    |  Identity  |  (28, 28, 128)  |      65536 / 0       |
| bn3d_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3d_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3d_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3d_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3d_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3d(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res4a_branch2a...|    (1, 1)     |   2    |  Identity  |  (14, 14, 256)  |      131072 / 0      |
| bn4a_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4a_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4a_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4a_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4a_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4a_branch1(...|    (1, 1)     |   2    |  Identity  | (14, 14, 1024)  |      524288 / 0      |
| bn4a_branch1(B...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4a(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4b_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4b_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4b_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4b_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4b_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4b_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4b(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4c_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4c_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4c_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4c_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4c_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4c_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4c(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4d_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4d_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4d_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4d_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4d_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4d_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4d(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4e_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4e_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4e_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4e_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4e_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4e_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4e(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4f_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4f_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4f_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4f_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4f_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4f_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4f(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res5a_branch2a...|    (1, 1)     |   2    |  Identity  |   (7, 7, 512)   |      524288 / 0      |
| bn5a_branch2a(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5a_branch2b...|    (3, 3)     |   1    |  Identity  |   (7, 7, 512)   |     2359296 / 0      |
| bn5a_branch2b(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5a_branch2c...|    (1, 1)     |   1    |  Identity  |  (7, 7, 2048)   |     1048576 / 0      |
| bn5a_branch2c(...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5a_branch1(...|    (1, 1)     |   2    |  Identity  |  (7, 7, 2048)   |     2097152 / 0      |
| bn5a_branch1(B...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5a(Resid.)    |     None      |  None  | Rectifier  |  (7, 7, 2048)   |        0 / 0         |
| res5b_branch2a...|    (1, 1)     |   1    |  Identity  |   (7, 7, 512)   |     1048576 / 0      |
| bn5b_branch2a(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5b_branch2b...|    (3, 3)     |   1    |  Identity  |   (7, 7, 512)   |     2359296 / 0      |
| bn5b_branch2b(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5b_branch2c...|    (1, 1)     |   1    |  Identity  |  (7, 7, 2048)   |     1048576 / 0      |
| bn5b_branch2c(...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5b(Resid.)    |     None      |  None  | Rectifier  |  (7, 7, 2048)   |        0 / 0         |
| res5c_branch2a...|    (1, 1)     |   1    |  Identity  |   (7, 7, 512)   |     1048576 / 0      |
| bn5c_branch2a(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5c_branch2b...|    (3, 3)     |   1    |  Identity  |   (7, 7, 512)   |     2359296 / 0      |
| bn5c_branch2b(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5c_branch2c...|    (1, 1)     |   1    |  Identity  |  (7, 7, 2048)   |     1048576 / 0      |
| bn5c_branch2c(...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5c(Resid.)    |     None      |  None  | Rectifier  |  (7, 7, 2048)   |        0 / 0         |
| pool5(Pool)      |    (7, 7)     |   7    |    Mean    |  (1, 1, 2048)   |        0 / 0         |
| fc1000(Output)   | (2048, 1000)  |  None  |  Softmax   |      1000       |    2048000 / 1000    |
*==================*===============*========*============*=================*======================*
|Total Number of Parameters: 25,557,096                                                           |
*=================================================================================================*

Note: 138 million parameters!!!

In [46]:
model2.plot_network()
Out[46]:
resnet50_caffe DAG for resnet50_caffe: data data (input) Input Size: (224, 224, 3) conv1 conv1 (convo) Kernel Size: Activation: Output Size: (7, 7) Identity (112, 112, 64) data->conv1 bn_conv1 bn_conv1 (batchnorm) Activation: Output Size: Rectifier (112, 112, 64) conv1->bn_conv1 pool1 pool1 (pool) Kernel Size: Output Size: (3, 3) (56, 56, 64) bn_conv1->pool1 res2a_branch2a res2a_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 64) pool1->res2a_branch2a res2a_branch1 res2a_branch1 (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 256) pool1->res2a_branch1 bn2a_branch2a bn2a_branch2a (batchnorm) Activation: Output Size: Rectifier (56, 56, 64) res2a_branch2a->bn2a_branch2a res2a_branch2b res2a_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (56, 56, 64) bn2a_branch2a->res2a_branch2b bn2a_branch2b bn2a_branch2b (batchnorm) Activation: Output Size: Rectifier (56, 56, 64) res2a_branch2b->bn2a_branch2b res2a_branch2c res2a_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 256) bn2a_branch2b->res2a_branch2c bn2a_branch2c bn2a_branch2c (batchnorm) Activation: Output Size: Identity (56, 56, 256) res2a_branch2c->bn2a_branch2c res2a res2a (residual) Activation: Output Size: Rectifier (56, 56, 256) bn2a_branch2c->res2a bn2a_branch1 bn2a_branch1 (batchnorm) Activation: Output Size: Identity (56, 56, 256) res2a_branch1->bn2a_branch1 bn2a_branch1->res2a res2b_branch2a res2b_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 64) res2a->res2b_branch2a res2b res2b (residual) Activation: Output Size: Rectifier (56, 56, 256) res2a->res2b bn2b_branch2a bn2b_branch2a (batchnorm) Activation: Output Size: Rectifier (56, 56, 64) res2b_branch2a->bn2b_branch2a res2b_branch2b res2b_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (56, 56, 64) bn2b_branch2a->res2b_branch2b bn2b_branch2b bn2b_branch2b (batchnorm) Activation: Output Size: Rectifier (56, 56, 64) res2b_branch2b->bn2b_branch2b res2b_branch2c res2b_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 256) bn2b_branch2b->res2b_branch2c bn2b_branch2c bn2b_branch2c (batchnorm) Activation: Output Size: Identity (56, 56, 256) res2b_branch2c->bn2b_branch2c bn2b_branch2c->res2b res2c_branch2a res2c_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 64) res2b->res2c_branch2a res2c res2c (residual) Activation: Output Size: Rectifier (56, 56, 256) res2b->res2c bn2c_branch2a bn2c_branch2a (batchnorm) Activation: Output Size: Rectifier (56, 56, 64) res2c_branch2a->bn2c_branch2a res2c_branch2b res2c_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (56, 56, 64) bn2c_branch2a->res2c_branch2b bn2c_branch2b bn2c_branch2b (batchnorm) Activation: Output Size: Rectifier (56, 56, 64) res2c_branch2b->bn2c_branch2b res2c_branch2c res2c_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (56, 56, 256) bn2c_branch2b->res2c_branch2c bn2c_branch2c bn2c_branch2c (batchnorm) Activation: Output Size: Identity (56, 56, 256) res2c_branch2c->bn2c_branch2c bn2c_branch2c->res2c res3a_branch2a res3a_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 128) res2c->res3a_branch2a res3a_branch1 res3a_branch1 (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 512) res2c->res3a_branch1 bn3a_branch2a bn3a_branch2a (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3a_branch2a->bn3a_branch2a res3a_branch2b res3a_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (28, 28, 128) bn3a_branch2a->res3a_branch2b bn3a_branch2b bn3a_branch2b (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3a_branch2b->bn3a_branch2b res3a_branch2c res3a_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 512) bn3a_branch2b->res3a_branch2c bn3a_branch2c bn3a_branch2c (batchnorm) Activation: Output Size: Identity (28, 28, 512) res3a_branch2c->bn3a_branch2c res3a res3a (residual) Activation: Output Size: Rectifier (28, 28, 512) bn3a_branch2c->res3a bn3a_branch1 bn3a_branch1 (batchnorm) Activation: Output Size: Identity (28, 28, 512) res3a_branch1->bn3a_branch1 bn3a_branch1->res3a res3b_branch2a res3b_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 128) res3a->res3b_branch2a res3b res3b (residual) Activation: Output Size: Rectifier (28, 28, 512) res3a->res3b bn3b_branch2a bn3b_branch2a (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3b_branch2a->bn3b_branch2a res3b_branch2b res3b_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (28, 28, 128) bn3b_branch2a->res3b_branch2b bn3b_branch2b bn3b_branch2b (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3b_branch2b->bn3b_branch2b res3b_branch2c res3b_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 512) bn3b_branch2b->res3b_branch2c bn3b_branch2c bn3b_branch2c (batchnorm) Activation: Output Size: Identity (28, 28, 512) res3b_branch2c->bn3b_branch2c bn3b_branch2c->res3b res3c_branch2a res3c_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 128) res3b->res3c_branch2a res3c res3c (residual) Activation: Output Size: Rectifier (28, 28, 512) res3b->res3c bn3c_branch2a bn3c_branch2a (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3c_branch2a->bn3c_branch2a res3c_branch2b res3c_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (28, 28, 128) bn3c_branch2a->res3c_branch2b bn3c_branch2b bn3c_branch2b (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3c_branch2b->bn3c_branch2b res3c_branch2c res3c_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 512) bn3c_branch2b->res3c_branch2c bn3c_branch2c bn3c_branch2c (batchnorm) Activation: Output Size: Identity (28, 28, 512) res3c_branch2c->bn3c_branch2c bn3c_branch2c->res3c res3d_branch2a res3d_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 128) res3c->res3d_branch2a res3d res3d (residual) Activation: Output Size: Rectifier (28, 28, 512) res3c->res3d bn3d_branch2a bn3d_branch2a (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3d_branch2a->bn3d_branch2a res3d_branch2b res3d_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (28, 28, 128) bn3d_branch2a->res3d_branch2b bn3d_branch2b bn3d_branch2b (batchnorm) Activation: Output Size: Rectifier (28, 28, 128) res3d_branch2b->bn3d_branch2b res3d_branch2c res3d_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (28, 28, 512) bn3d_branch2b->res3d_branch2c bn3d_branch2c bn3d_branch2c (batchnorm) Activation: Output Size: Identity (28, 28, 512) res3d_branch2c->bn3d_branch2c bn3d_branch2c->res3d res4a_branch2a res4a_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 256) res3d->res4a_branch2a res4a_branch1 res4a_branch1 (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) res3d->res4a_branch1 bn4a_branch2a bn4a_branch2a (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4a_branch2a->bn4a_branch2a res4a_branch2b res4a_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (14, 14, 256) bn4a_branch2a->res4a_branch2b bn4a_branch2b bn4a_branch2b (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4a_branch2b->bn4a_branch2b res4a_branch2c res4a_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) bn4a_branch2b->res4a_branch2c bn4a_branch2c bn4a_branch2c (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4a_branch2c->bn4a_branch2c res4a res4a (residual) Activation: Output Size: Rectifier (14, 14, 1024) bn4a_branch2c->res4a bn4a_branch1 bn4a_branch1 (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4a_branch1->bn4a_branch1 bn4a_branch1->res4a res4b_branch2a res4b_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 256) res4a->res4b_branch2a res4b res4b (residual) Activation: Output Size: Rectifier (14, 14, 1024) res4a->res4b bn4b_branch2a bn4b_branch2a (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4b_branch2a->bn4b_branch2a res4b_branch2b res4b_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (14, 14, 256) bn4b_branch2a->res4b_branch2b bn4b_branch2b bn4b_branch2b (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4b_branch2b->bn4b_branch2b res4b_branch2c res4b_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) bn4b_branch2b->res4b_branch2c bn4b_branch2c bn4b_branch2c (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4b_branch2c->bn4b_branch2c bn4b_branch2c->res4b res4c_branch2a res4c_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 256) res4b->res4c_branch2a res4c res4c (residual) Activation: Output Size: Rectifier (14, 14, 1024) res4b->res4c bn4c_branch2a bn4c_branch2a (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4c_branch2a->bn4c_branch2a res4c_branch2b res4c_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (14, 14, 256) bn4c_branch2a->res4c_branch2b bn4c_branch2b bn4c_branch2b (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4c_branch2b->bn4c_branch2b res4c_branch2c res4c_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) bn4c_branch2b->res4c_branch2c bn4c_branch2c bn4c_branch2c (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4c_branch2c->bn4c_branch2c bn4c_branch2c->res4c res4d_branch2a res4d_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 256) res4c->res4d_branch2a res4d res4d (residual) Activation: Output Size: Rectifier (14, 14, 1024) res4c->res4d bn4d_branch2a bn4d_branch2a (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4d_branch2a->bn4d_branch2a res4d_branch2b res4d_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (14, 14, 256) bn4d_branch2a->res4d_branch2b bn4d_branch2b bn4d_branch2b (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4d_branch2b->bn4d_branch2b res4d_branch2c res4d_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) bn4d_branch2b->res4d_branch2c bn4d_branch2c bn4d_branch2c (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4d_branch2c->bn4d_branch2c bn4d_branch2c->res4d res4e_branch2a res4e_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 256) res4d->res4e_branch2a res4e res4e (residual) Activation: Output Size: Rectifier (14, 14, 1024) res4d->res4e bn4e_branch2a bn4e_branch2a (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4e_branch2a->bn4e_branch2a res4e_branch2b res4e_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (14, 14, 256) bn4e_branch2a->res4e_branch2b bn4e_branch2b bn4e_branch2b (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4e_branch2b->bn4e_branch2b res4e_branch2c res4e_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) bn4e_branch2b->res4e_branch2c bn4e_branch2c bn4e_branch2c (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4e_branch2c->bn4e_branch2c bn4e_branch2c->res4e res4f_branch2a res4f_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 256) res4e->res4f_branch2a res4f res4f (residual) Activation: Output Size: Rectifier (14, 14, 1024) res4e->res4f bn4f_branch2a bn4f_branch2a (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4f_branch2a->bn4f_branch2a res4f_branch2b res4f_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (14, 14, 256) bn4f_branch2a->res4f_branch2b bn4f_branch2b bn4f_branch2b (batchnorm) Activation: Output Size: Rectifier (14, 14, 256) res4f_branch2b->bn4f_branch2b res4f_branch2c res4f_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (14, 14, 1024) bn4f_branch2b->res4f_branch2c bn4f_branch2c bn4f_branch2c (batchnorm) Activation: Output Size: Identity (14, 14, 1024) res4f_branch2c->bn4f_branch2c bn4f_branch2c->res4f res5a_branch2a res5a_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 512) res4f->res5a_branch2a res5a_branch1 res5a_branch1 (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 2048) res4f->res5a_branch1 bn5a_branch2a bn5a_branch2a (batchnorm) Activation: Output Size: Rectifier (7, 7, 512) res5a_branch2a->bn5a_branch2a res5a_branch2b res5a_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (7, 7, 512) bn5a_branch2a->res5a_branch2b bn5a_branch2b bn5a_branch2b (batchnorm) Activation: Output Size: Rectifier (7, 7, 512) res5a_branch2b->bn5a_branch2b res5a_branch2c res5a_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 2048) bn5a_branch2b->res5a_branch2c bn5a_branch2c bn5a_branch2c (batchnorm) Activation: Output Size: Identity (7, 7, 2048) res5a_branch2c->bn5a_branch2c res5a res5a (residual) Activation: Output Size: Rectifier (7, 7, 2048) bn5a_branch2c->res5a bn5a_branch1 bn5a_branch1 (batchnorm) Activation: Output Size: Identity (7, 7, 2048) res5a_branch1->bn5a_branch1 bn5a_branch1->res5a res5b_branch2a res5b_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 512) res5a->res5b_branch2a res5b res5b (residual) Activation: Output Size: Rectifier (7, 7, 2048) res5a->res5b bn5b_branch2a bn5b_branch2a (batchnorm) Activation: Output Size: Rectifier (7, 7, 512) res5b_branch2a->bn5b_branch2a res5b_branch2b res5b_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (7, 7, 512) bn5b_branch2a->res5b_branch2b bn5b_branch2b bn5b_branch2b (batchnorm) Activation: Output Size: Rectifier (7, 7, 512) res5b_branch2b->bn5b_branch2b res5b_branch2c res5b_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 2048) bn5b_branch2b->res5b_branch2c bn5b_branch2c bn5b_branch2c (batchnorm) Activation: Output Size: Identity (7, 7, 2048) res5b_branch2c->bn5b_branch2c bn5b_branch2c->res5b res5c_branch2a res5c_branch2a (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 512) res5b->res5c_branch2a res5c res5c (residual) Activation: Output Size: Rectifier (7, 7, 2048) res5b->res5c bn5c_branch2a bn5c_branch2a (batchnorm) Activation: Output Size: Rectifier (7, 7, 512) res5c_branch2a->bn5c_branch2a res5c_branch2b res5c_branch2b (convo) Kernel Size: Activation: Output Size: (3, 3) Identity (7, 7, 512) bn5c_branch2a->res5c_branch2b bn5c_branch2b bn5c_branch2b (batchnorm) Activation: Output Size: Rectifier (7, 7, 512) res5c_branch2b->bn5c_branch2b res5c_branch2c res5c_branch2c (convo) Kernel Size: Activation: Output Size: (1, 1) Identity (7, 7, 2048) bn5c_branch2b->res5c_branch2c bn5c_branch2c bn5c_branch2c (batchnorm) Activation: Output Size: Identity (7, 7, 2048) res5c_branch2c->bn5c_branch2c bn5c_branch2c->res5c pool5 pool5 (pool) Kernel Size: Output Size: (7, 7) (1, 1, 2048) res5c->pool5 fc1000 fc1000 (output) Kernel Size: Activation: Output Size: (2048, 1000) Softmax 1000 pool5->fc1000

Scoring an image of Kerboard

In [47]:
new_img = ImageTable.load_files(sess,path='/dept/cas/leliuz/Data/Demo/Computer_Kerboard')
In [48]:
new_img.resize(224)
In [49]:
new_img.show()
In [50]:
model2.predict(data=new_img)
Out[50]:
§ ScoreInfo
Descr Value
0 Number of Observations Read 1
1 Number of Observations Used 1
2 Misclassification Error (%) 100
3 Loss Error 0
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Valid_Res_fEkwmV 1 1006 CASTable('Valid_Res_fEkwmV', caslib='CASUSERHD...

elapsed 6.94s · user 10s · sys 37.6s · mem 2.75e+04MB

In [51]:
model2.plot_predict_res(image_id=0,type='A')

Score the test image in the example

In [52]:
model2.predict(data=te_img)
Out[52]:
§ ScoreInfo
Descr Value
0 Number of Observations Read 81
1 Number of Observations Used 81
2 Misclassification Error (%) 100
3 Loss Error 0
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Valid_Res_Yso5Rt 81 1006 CASTable('Valid_Res_Yso5Rt', caslib='CASUSERHD...

elapsed 5.98s · user 45.9s · sys 78s · mem 5.05e+04MB

In [53]:
model2.plot_predict_res(image_id=1)

Model3: ImageNet VGG16

In [54]:
model3 = VGG16(sess, model_name='VGG16', 
               n_classes=1000, n_channels=3, width=224, height=224, scale=1,
               random_flip='none', random_crop='none',
               offsets=(103.939, 116.779, 123.68),
               pre_train_weight=True, include_top=True)

NOTE: Model table is attached successfully!
NOTE: Model is named to "vgg16" according to the model name in the table.
NOTE: Cloud Analytic Services made the uploaded file available as table LABEL_2LBZXY in caslib CASUSERHDFS(leliuz).
NOTE: The table LABEL_2LBZXY has been created in caslib CASUSERHDFS(leliuz) from binary data uploaded to Cloud Analytic Services.
In [55]:
model3.print_summary()

*==================*===============*========*============*=================*======================*
|   Layer (Type)   |  Kernel Size  | Stride | Activation |   Output Size   | Number of Parameters |
*------------------*---------------*--------*------------*-----------------*----------------------*
| data(Input)      |     None      |  None  |    None    |  (224, 224, 3)  |        0 / 0         |
| conv1_1(Convo.)  |    (3, 3)     |   1    | Rectifier  | (224, 224, 64)  |      1728 / 64       |
| conv1_2(Convo.)  |    (3, 3)     |   1    | Rectifier  | (224, 224, 64)  |      36864 / 64      |
| pool1(Pool)      |    (2, 2)     |   2    |    Max     | (112, 112, 64)  |        0 / 0         |
| conv2_1(Convo.)  |    (3, 3)     |   1    | Rectifier  | (112, 112, 128) |     73728 / 128      |
| conv2_2(Convo.)  |    (3, 3)     |   1    | Rectifier  | (112, 112, 128) |     147456 / 128     |
| pool2(Pool)      |    (2, 2)     |   2    |    Max     |  (56, 56, 128)  |        0 / 0         |
| conv3_1(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (56, 56, 256)  |     294912 / 256     |
| conv3_2(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (56, 56, 256)  |     589824 / 256     |
| conv3_3(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (56, 56, 256)  |     589824 / 256     |
| pool3(Pool)      |    (2, 2)     |   2    |    Max     |  (28, 28, 256)  |        0 / 0         |
| conv4_1(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (28, 28, 512)  |    1179648 / 512     |
| conv4_2(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (28, 28, 512)  |    2359296 / 512     |
| conv4_3(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (28, 28, 512)  |    2359296 / 512     |
| pool4(Pool)      |    (2, 2)     |   2    |    Max     |  (14, 14, 512)  |        0 / 0         |
| conv5_1(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (14, 14, 512)  |    2359296 / 512     |
| conv5_2(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (14, 14, 512)  |    2359296 / 512     |
| conv5_3(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (14, 14, 512)  |    2359296 / 512     |
| pool5(Pool)      |    (2, 2)     |   2    |    Max     |   (7, 7, 512)   |        0 / 0         |
| fc6(F.C.)        | (25088, 4096) |  None  | Rectifier  |      4096       |   102760448 / 4096   |
| fc7(F.C.)        | (4096, 4096)  |  None  | Rectifier  |      4096       |   16777216 / 4096    |
| fc8(Output)      | (4096, 1000)  |  None  |  Softmax   |      1000       |    4096000 / 1000    |
*==================*===============*========*============*=================*======================*
|Total Number of Parameters: 138,357,544                                                          |
*=================================================================================================*

Extract linear feture using ImageNet VGG16

In [56]:
X, y = model3.get_features(data=te_img, dense_layer='pool5')
In [57]:
X, y
Out[57]:
(array([[0.        , 0.        , 0.        , ..., 5.82558489, 0.        ,
         0.        ],
        [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
         0.        ],
        [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
         0.        ],
        ...,
        [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
         0.        ],
        [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
         0.        ],
        [0.        , 0.        , 0.        , ..., 0.        , 0.        ,
         0.        ]]),
 array(['Giraffe', 'Dolphin', 'Dolphin', 'Giraffe', 'Giraffe', 'Dolphin',
        'Giraffe', 'Giraffe', 'Dolphin', 'Dolphin', 'Giraffe', 'Dolphin',
        'Dolphin', 'Dolphin', 'Dolphin', 'Giraffe', 'Dolphin', 'Dolphin',
        'Dolphin', 'Dolphin', 'Giraffe', 'Dolphin', 'Dolphin', 'Giraffe',
        'Dolphin', 'Dolphin', 'Dolphin', 'Dolphin', 'Dolphin', 'Giraffe',
        'Dolphin', 'Dolphin', 'Giraffe', 'Dolphin', 'Giraffe', 'Giraffe',
        'Dolphin', 'Giraffe', 'Dolphin', 'Dolphin', 'Dolphin', 'Giraffe',
        'Giraffe', 'Giraffe', 'Dolphin', 'Dolphin', 'Dolphin', 'Dolphin',
        'Giraffe', 'Dolphin', 'Dolphin', 'Giraffe', 'Giraffe', 'Giraffe',
        'Giraffe', 'Dolphin', 'Giraffe', 'Giraffe', 'Giraffe', 'Giraffe',
        'Giraffe', 'Dolphin', 'Dolphin', 'Dolphin', 'Dolphin', 'Dolphin',
        'Giraffe', 'Dolphin', 'Giraffe', 'Giraffe', 'Giraffe', 'Dolphin',
        'Dolphin', 'Dolphin', 'Dolphin', 'Giraffe', 'Dolphin', 'Giraffe',
        'Dolphin', 'Dolphin', 'Giraffe'], dtype=object))

Fit a descision tree using Pipefitter

In [58]:
from pipefitter.model_selection import HyperParameterTuning
from pipefitter.estimator import DecisionTree, DecisionForest, GBTree
import pandas as pd
In [59]:
params = dict(target='label', 
              inputs=[str(i) for i in range(16)])
dtree = DecisionTree(max_depth=6, **params)
data = pd.DataFrame(X)
data['label']=y
casdata = sess.upload_frame(data)
model = dtree.fit(casdata)
score = model.score(casdata)
score

NOTE: Cloud Analytic Services made the uploaded file available as table TMPKD_ROG4P in caslib CASUSERHDFS(leliuz).
NOTE: The table TMPKD_ROG4P has been created in caslib CASUSERHDFS(leliuz) from binary data uploaded to Cloud Analytic Services.
Out[59]:
NTreeNodes                         5
MaxNBranches                       2
NLevels                            3
NLeaves                            3
NBins                             20
MinimumSizeofLeaves                5
MaximumSizeofLeaves               71
NVariables                        16
ConfidenceLevelforPruning       0.25
NObsRead                          81
NObsUsed                          81
MisClassificationRate        40.7407
dtype: object

Fit a PCA model using sklearn

In [60]:
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
In [61]:
X_r = PCA(n_components=2).fit_transform(X)
plt.scatter(X_r[y=='Giraffe',0], X_r[y=='Giraffe',1],marker='^',c='g',label='Giraffe')
plt.scatter(X_r[y=='Dolphin',0], X_r[y=='Dolphin',1],marker='o',c='b',label='Dolphin')
plt.legend(bbox_to_anchor=(0.01, 0.99), loc=2, borderaxespad=0.)
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.show()

Model4: Transfer Learning with ImageNet ResNet50

In [62]:
model4 = ResNet50_Caffe(sess, model_name='ResNet50_Caffe', 
                n_classes=2, n_channels=3, width=224, height=224, scale=1,
                random_flip='none', random_crop='none',
                offsets=tr_img.channel_means,
                pre_train_weight=True, include_top=False)

NOTE: Model table is attached successfully!
NOTE: Model is named to "resnet50_caffe" according to the model name in the table.
In [63]:
model4.print_summary()

*==================*===============*========*============*=================*======================*
|   Layer (Type)   |  Kernel Size  | Stride | Activation |   Output Size   | Number of Parameters |
*------------------*---------------*--------*------------*-----------------*----------------------*
| data(Input)      |     None      |  None  |    None    |  (224, 224, 3)  |        0 / 0         |
| conv1(Convo.)    |    (7, 7)     |   2    |  Identity  | (112, 112, 64)  |      9408 / 64       |
| bn_conv1(B.N.)   |     None      |  None  | Rectifier  | (112, 112, 64)  |       0 / 128        |
| pool1(Pool)      |    (3, 3)     |   2    |    Max     |  (56, 56, 64)   |        0 / 0         |
| res2a_branch2a...|    (1, 1)     |   1    |  Identity  |  (56, 56, 64)   |       4096 / 0       |
| bn2a_branch2a(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2a_branch2b...|    (3, 3)     |   1    |  Identity  |  (56, 56, 64)   |      36864 / 0       |
| bn2a_branch2b(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2a_branch2c...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2a_branch2c(...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2a_branch1(...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2a_branch1(B...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2a(Resid.)    |     None      |  None  | Rectifier  |  (56, 56, 256)  |        0 / 0         |
| res2b_branch2a...|    (1, 1)     |   1    |  Identity  |  (56, 56, 64)   |      16384 / 0       |
| bn2b_branch2a(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2b_branch2b...|    (3, 3)     |   1    |  Identity  |  (56, 56, 64)   |      36864 / 0       |
| bn2b_branch2b(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2b_branch2c...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2b_branch2c(...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2b(Resid.)    |     None      |  None  | Rectifier  |  (56, 56, 256)  |        0 / 0         |
| res2c_branch2a...|    (1, 1)     |   1    |  Identity  |  (56, 56, 64)   |      16384 / 0       |
| bn2c_branch2a(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2c_branch2b...|    (3, 3)     |   1    |  Identity  |  (56, 56, 64)   |      36864 / 0       |
| bn2c_branch2b(...|     None      |  None  | Rectifier  |  (56, 56, 64)   |       0 / 128        |
| res2c_branch2c...|    (1, 1)     |   1    |  Identity  |  (56, 56, 256)  |      16384 / 0       |
| bn2c_branch2c(...|     None      |  None  |  Identity  |  (56, 56, 256)  |       0 / 512        |
| res2c(Resid.)    |     None      |  None  | Rectifier  |  (56, 56, 256)  |        0 / 0         |
| res3a_branch2a...|    (1, 1)     |   2    |  Identity  |  (28, 28, 128)  |      32768 / 0       |
| bn3a_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3a_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3a_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3a_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3a_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3a_branch1(...|    (1, 1)     |   2    |  Identity  |  (28, 28, 512)  |      131072 / 0      |
| bn3a_branch1(B...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3a(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res3b_branch2a...|    (1, 1)     |   1    |  Identity  |  (28, 28, 128)  |      65536 / 0       |
| bn3b_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3b_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3b_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3b_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3b_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3b(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res3c_branch2a...|    (1, 1)     |   1    |  Identity  |  (28, 28, 128)  |      65536 / 0       |
| bn3c_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3c_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3c_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3c_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3c_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3c(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res3d_branch2a...|    (1, 1)     |   1    |  Identity  |  (28, 28, 128)  |      65536 / 0       |
| bn3d_branch2a(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3d_branch2b...|    (3, 3)     |   1    |  Identity  |  (28, 28, 128)  |      147456 / 0      |
| bn3d_branch2b(...|     None      |  None  | Rectifier  |  (28, 28, 128)  |       0 / 256        |
| res3d_branch2c...|    (1, 1)     |   1    |  Identity  |  (28, 28, 512)  |      65536 / 0       |
| bn3d_branch2c(...|     None      |  None  |  Identity  |  (28, 28, 512)  |       0 / 1024       |
| res3d(Resid.)    |     None      |  None  | Rectifier  |  (28, 28, 512)  |        0 / 0         |
| res4a_branch2a...|    (1, 1)     |   2    |  Identity  |  (14, 14, 256)  |      131072 / 0      |
| bn4a_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4a_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4a_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4a_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4a_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4a_branch1(...|    (1, 1)     |   2    |  Identity  | (14, 14, 1024)  |      524288 / 0      |
| bn4a_branch1(B...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4a(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4b_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4b_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4b_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4b_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4b_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4b_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4b(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4c_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4c_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4c_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4c_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4c_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4c_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4c(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4d_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4d_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4d_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4d_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4d_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4d_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4d(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4e_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4e_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4e_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4e_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4e_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4e_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4e(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res4f_branch2a...|    (1, 1)     |   1    |  Identity  |  (14, 14, 256)  |      262144 / 0      |
| bn4f_branch2a(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4f_branch2b...|    (3, 3)     |   1    |  Identity  |  (14, 14, 256)  |      589824 / 0      |
| bn4f_branch2b(...|     None      |  None  | Rectifier  |  (14, 14, 256)  |       0 / 512        |
| res4f_branch2c...|    (1, 1)     |   1    |  Identity  | (14, 14, 1024)  |      262144 / 0      |
| bn4f_branch2c(...|     None      |  None  |  Identity  | (14, 14, 1024)  |       0 / 2048       |
| res4f(Resid.)    |     None      |  None  | Rectifier  | (14, 14, 1024)  |        0 / 0         |
| res5a_branch2a...|    (1, 1)     |   2    |  Identity  |   (7, 7, 512)   |      524288 / 0      |
| bn5a_branch2a(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5a_branch2b...|    (3, 3)     |   1    |  Identity  |   (7, 7, 512)   |     2359296 / 0      |
| bn5a_branch2b(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5a_branch2c...|    (1, 1)     |   1    |  Identity  |  (7, 7, 2048)   |     1048576 / 0      |
| bn5a_branch2c(...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5a_branch1(...|    (1, 1)     |   2    |  Identity  |  (7, 7, 2048)   |     2097152 / 0      |
| bn5a_branch1(B...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5a(Resid.)    |     None      |  None  | Rectifier  |  (7, 7, 2048)   |        0 / 0         |
| res5b_branch2a...|    (1, 1)     |   1    |  Identity  |   (7, 7, 512)   |     1048576 / 0      |
| bn5b_branch2a(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5b_branch2b...|    (3, 3)     |   1    |  Identity  |   (7, 7, 512)   |     2359296 / 0      |
| bn5b_branch2b(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5b_branch2c...|    (1, 1)     |   1    |  Identity  |  (7, 7, 2048)   |     1048576 / 0      |
| bn5b_branch2c(...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5b(Resid.)    |     None      |  None  | Rectifier  |  (7, 7, 2048)   |        0 / 0         |
| res5c_branch2a...|    (1, 1)     |   1    |  Identity  |   (7, 7, 512)   |     1048576 / 0      |
| bn5c_branch2a(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5c_branch2b...|    (3, 3)     |   1    |  Identity  |   (7, 7, 512)   |     2359296 / 0      |
| bn5c_branch2b(...|     None      |  None  | Rectifier  |   (7, 7, 512)   |       0 / 1024       |
| res5c_branch2c...|    (1, 1)     |   1    |  Identity  |  (7, 7, 2048)   |     1048576 / 0      |
| bn5c_branch2c(...|     None      |  None  |  Identity  |  (7, 7, 2048)   |       0 / 4096       |
| res5c(Resid.)    |     None      |  None  | Rectifier  |  (7, 7, 2048)   |        0 / 0         |
| pool5(Pool)      |    (7, 7)     |   7    |    Mean    |  (1, 1, 2048)   |        0 / 0         |
| output(Output)   |   (2048, 2)   |  None  |  Softmax   |        2        |       4096 / 2       |
*==================*===============*========*============*=================*======================*
|Total Number of Parameters: 23,512,194                                                           |
*=================================================================================================*
In [64]:
model4.fit(data=tr_img, mini_batch_size=1, max_epochs=3, lr=5E-3)

NOTE: Training based on existing weights.
WARNING: Source layer conv1 to batch normalization layer bn_conv1 includes a bias term.
NOTE:  The Synchronous mode is enabled.
NOTE:  The total number of parameters is 23512194.
NOTE:  The approximate memory cost is 10082.00 MB.
NOTE:  Loading weights cost       1.44 (s).
NOTE:  Initializing each layer cost       4.76 (s).
NOTE:  The total number of workers is 7.
NOTE:  The total number of threads on each worker is 32.
NOTE:  The total minibatch size per thread on each worker is 1.
NOTE:  The maximum minibatch size across all workers for the synchronous mode is 224.
NOTE:  Target variable: _label_
NOTE:  Number of levels for the target variable:      2
NOTE:  Levels for the target variable:
NOTE:  Level      0: Dolphin
NOTE:  Level      1: Giraffe
NOTE:  Number of input variables:     1
NOTE:  Number of numeric input variables:      1
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              224   0.005          0.8449      0.5982         1.35
NOTE:        1              224   0.005          0.5667      0.2902         1.27
NOTE:        2              224   0.005          0.4704      0.1696         1.23
NOTE:        3              224   0.005          0.3423      0.0714         1.26
NOTE:        4              224   0.005          0.2795      0.0491         1.23
NOTE:        5              158   0.005          0.2492      0.0316         1.23
NOTE:        6               32   0.005          0.2578      0.0625         1.19
NOTE:        7                2   0.005          0.1483           0         1.24
NOTE:  Epoch           Learning Rate     Loss    Fit Error      Time (s)
NOTE:          0           0.005        0.464     0.2066        94.34
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              224   0.005          0.1655      0.0045         1.28
NOTE:        1              224   0.005           0.147           0         1.27
NOTE:        2              224   0.005          0.1421      0.0179         1.22
NOTE:        3              224   0.005          0.1274           0         1.24
NOTE:        4              224   0.005          0.1064           0         1.26
NOTE:        5              158   0.005          0.1145           0         1.24
NOTE:        6               32   0.005          0.0935           0         1.26
NOTE:        7                2   0.005          0.0173           0         1.26
NOTE:          1           0.005       0.1336     0.0038        78.59
NOTE:  Batch              nUsed   Learning Rate     Loss    Fit Error      Time (s) (Training)
NOTE:        0              224   0.005          0.0932           0         1.27
NOTE:        1              224   0.005          0.0834           0         1.25
NOTE:        2              224   0.005          0.0832      0.0089         1.25
NOTE:        3              224   0.005          0.0786           0         1.24
NOTE:        4              224   0.005          0.0659           0         1.23
NOTE:        5              158   0.005          0.0762           0         1.22
NOTE:        6               32   0.005          0.0548           0         1.24
NOTE:        7                2   0.005          0.0095           0         1.22
NOTE:          2           0.005       0.0795     0.0015        74.68
NOTE:  The optimization reached the maximum number of epochs.
NOTE:  The total time is     247.61 (s).
Out[64]:
§ ModelInfo
Descr Value
0 Model Name resnet50_caffe
1 Model Type Convolutional Neural Network
2 Number of Layers 126
3 Number of Input Layers 1
4 Number of Output Layers 1
5 Number of Convolutional Layers 53
6 Number of Pooling Layers 2
7 Number of Fully Connected Layers 0
8 Number of Batch Normalization Layers 53
9 Number of Residual Layers 16
10 Number of Weight Parameters 23459008
11 Number of Bias Parameters 53186
12 Total Number of Model Parameters 23512194
13 Approximate Memory Cost for Training (MB) 10082
§ OptIterHistory
Epoch LearningRate Loss FitError
0 1 0.005 0.464024 0.206555
1 2 0.005 0.133627 0.003811
2 3 0.005 0.079538 0.001524
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) resnet50_caffe_weights 23565314 3 CASTable('resnet50_caffe_weights', caslib='CAS...

elapsed 256s · user 1.87e+04s · sys 1.47e+03s · mem 7.17e+04MB

In [65]:
model4.training_history
Out[65]:
Epoch LearningRate Loss FitError
0 1 0.005 0.464024 0.206555
1 2 0.005 0.133627 0.003811
2 3 0.005 0.079538 0.001524
In [66]:
model4.plot_training_history(fig_size=(10,4))

Scoring

In [67]:
model4.predict(data=te_img)
Out[67]:
§ ScoreInfo
Descr Value
0 Number of Observations Read 81
1 Number of Observations Used 81
2 Misclassification Error (%) 0
3 Loss Error 0.076925
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Valid_Res_ltODHE 81 8 CASTable('Valid_Res_ltODHE', caslib='CASUSERHD...

elapsed 4.73s · user 41.5s · sys 82.3s · mem 5.05e+04MB

In [68]:
model4.valid_conf_mat
Out[68]:
§ Crosstab
_label_ Col1 Col2
0 Dolphin 47.0 0.0
1 Giraffe 0.0 34.0

elapsed 0.0317s · user 0.0521s · sys 0.0606s · mem 13MB

In [69]:
model4.plot_predict_res(type='C',image_id=2)
In [70]:
model4.plot_predict_res(type='A',image_id=0)

Heatmap analysis

In [72]:
model4.heat_map_analysis(data=te_img, mask_width=56, mask_height=56, step_size=8)

NOTE : The number of images in the table is too large, only 5 randomly selected images are used in analysis.
NOTE: Table IMAGEDATA_ICQFYM contains compressed images.
Out[72]:
_filename_0 _image_ _label_ heat_map
0 giraffe_11069.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Giraffe [[0.7036072015762329, 0.7036072015762329, 0.70...
1 dolphin_10191.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.9894927740097046, 0.9894927740097046, 0.98...
2 giraffe_10831.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Giraffe [[0.9486960768699646, 0.9486960768699646, 0.94...
3 dolphin_10937.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.9344797730445862, 0.9344797730445862, 0.93...
4 dolphin_10368.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.991266667842865, 0.991266667842865, 0.9912...

Model5: pretrained VGG16 model

In [73]:
model5 = Model(sess)
model5.load(path='/dept/cas/leliuz/Data/Demo/vgg16.sashdat')

NOTE: Model table is loaded successfully!
NOTE: Model is renamed to "vgg16" according to the model name in the table.
NOTE: vgg16_weights.sashdat is used as model weigths.
NOTE: Model weights attached successfully!
NOTE: vgg16_weights_attr.sashdat is used as weigths attribute.
NOTE: Model attributes attached successfully!
In [74]:
model5.print_summary()

*==================*===============*========*============*=================*======================*
|   Layer (Type)   |  Kernel Size  | Stride | Activation |   Output Size   | Number of Parameters |
*------------------*---------------*--------*------------*-----------------*----------------------*
| data(Input)      |     None      |  None  |    None    |  (224, 224, 3)  |        0 / 0         |
| conv1_1(Convo.)  |    (3, 3)     |   1    | Rectifier  | (224, 224, 64)  |      1728 / 64       |
| conv1_2(Convo.)  |    (3, 3)     |   1    | Rectifier  | (224, 224, 64)  |      36864 / 64      |
| pool1(Pool)      |    (2, 2)     |   2    |    Max     | (112, 112, 64)  |        0 / 0         |
| conv2_1(Convo.)  |    (3, 3)     |   1    | Rectifier  | (112, 112, 128) |     73728 / 128      |
| conv2_2(Convo.)  |    (3, 3)     |   1    | Rectifier  | (112, 112, 128) |     147456 / 128     |
| pool2(Pool)      |    (2, 2)     |   2    |    Max     |  (56, 56, 128)  |        0 / 0         |
| conv3_1(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (56, 56, 256)  |     294912 / 256     |
| conv3_2(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (56, 56, 256)  |     589824 / 256     |
| conv3_3(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (56, 56, 256)  |     589824 / 256     |
| pool3(Pool)      |    (2, 2)     |   2    |    Max     |  (28, 28, 256)  |        0 / 0         |
| conv4_1(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (28, 28, 512)  |    1179648 / 512     |
| conv4_2(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (28, 28, 512)  |    2359296 / 512     |
| conv4_3(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (28, 28, 512)  |    2359296 / 512     |
| pool4(Pool)      |    (2, 2)     |   2    |    Max     |  (14, 14, 512)  |        0 / 0         |
| conv5_1(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (14, 14, 512)  |    2359296 / 512     |
| conv5_2(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (14, 14, 512)  |    2359296 / 512     |
| conv5_3(Convo.)  |    (3, 3)     |   1    | Rectifier  |  (14, 14, 512)  |    2359296 / 512     |
| pool5(Pool)      |    (2, 2)     |   2    |    Max     |   (7, 7, 512)   |        0 / 0         |
| fc6(F.C.)        | (25088, 4096) |  None  | Rectifier  |      4096       |   102760448 / 4096   |
| fc7(F.C.)        | (4096, 4096)  |  None  | Rectifier  |      4096       |   16777216 / 4096    |
| fc8(Output)      |   (4096, 2)   |  None  |  Softmax   |        2        |       8192 / 2       |
*==================*===============*========*============*=================*======================*
|Total Number of Parameters: 134,268,738                                                          |
*=================================================================================================*

Scoring

In [75]:
model5.predict(te_img)
Out[75]:
§ ScoreInfo
Descr Value
0 Number of Observations Read 81
1 Number of Observations Used 81
2 Misclassification Error (%) 0
3 Loss Error 0.002312
§ OutputCasTables
casLib Name Rows Columns casTable
0 CASUSERHDFS(leliuz) Valid_Res_wXRh58 81 8 CASTable('Valid_Res_wXRh58', caslib='CASUSERHD...

elapsed 12s · user 153s · sys 37.1s · mem 6.03e+04MB

In [76]:
model5.valid_conf_mat
Out[76]:
§ Crosstab
_label_ Col1 Col2
0 Dolphin 47.0 0.0
1 Giraffe 0.0 34.0

elapsed 0.031s · user 0.0533s · sys 0.0744s · mem 13.1MB

In [77]:
model5.plot_predict_res(type='C',image_id=2)
In [78]:
model5.plot_predict_res(type='A',image_id=0)
In [79]:
model5.heat_map_analysis(data=te_img, mask_width=56, mask_height=56, step_size=8)

NOTE : The number of images in the table is too large, only 5 randomly selected images are used in analysis.
NOTE: Table IMAGEDATA_P4UF91 contains compressed images.
Out[79]:
_filename_0 _image_ _label_ heat_map
0 dolphin_10442.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.9998576641082764, 0.9998576641082764, 0.99...
1 giraffe_11070.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Giraffe [[0.999992847442627, 0.999992847442627, 0.9999...
2 giraffe_10446.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Giraffe [[0.9999914169311523, 0.9999914169311523, 0.99...
3 dolphin_10368.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.9997968077659607, 0.9997968077659607, 0.99...
4 dolphin_10811.jpg <PIL.JpegImagePlugin.JpegImageFile image mode=... Dolphin [[0.9928036332130432, 0.9928036332130432, 0.99...

View feature maps

In [80]:
model5.get_feature_maps(data=te_img,label='Dolphin', image_id=0)
In [81]:
model5.feature_maps.display(layer_id=0)
In [82]:
model5.feature_maps.display(layer_id=1)
In [83]:
model5.feature_maps.display(layer_id=2)
In [84]:
model5.feature_maps.display(layer_id=18)

NOTE: The maximum number of filters to be displayed is 64.
NOTE: Only the first 64 filters are displayed.
In [85]:
sess.endsession()
Out[85]:

elapsed 0.00747s · user 0.00235s · sys 0.0115s · mem 1.63MB

In [ ]: